Search engine systems commonly employ a collection of index servers to process a query submitted by an end user. In operation, a search engine system routes the user's query to a subset of the index servers. Each index server in the subset uses index information to identify the top n documents that are considered relevant to the query. Other processing modules in the search engine system then aggregate the results provided by the individual index servers to identify a final set of top m relevant documents. The search engine system then determines snippets and universal resource locator identifiers (URLs) corresponding to the m documents. The search engine system forwards those snippets and URLs to the end user.
In addition to the normal-use production traffic described above, an analyst may submit one or more experimental queries to the search engine system. Upon receiving search results, the analyst may use one or more metrics to assess the relevance of the search results, or to examine some other characteristic of the search results. Based on this investigation, the analyst can modify the behavior of the search engine system to improve its performance.
To perform the above-described analysis tasks, the analyst may submit queries and receive search results via the same data paths used by end users. For instance, the search engine system may generate results in response to an experimental query by aggregating the partial results provided by individual index servers. This approach, however, is not fully satisfactory for reasons set forth in the Detailed Description below.